Maximum Entropy Spectral Models for Color Constancy in the Presence of Interreflections

Interreflections in a scene can be exploited to improve upon surface and illuminant spectral estimation. In this paper, we present a novel maximum entropy approach to spectral color constancy in the presence of interreflections. Previous approaches employ linear model representations of surface and illuminant spectra. Such representations are not always practical as a database of spectra has to be specified in advance in the corresponding algorithms. The proposed approach has a major advantage over previous algorithms in that it requires only camera sensor responses from the mutual illumination or edge region of a folded surface and from the far from the edge regions to estimate surface and illuminant spectra. We demonstrate the feasibility of the approach while assuming a one-bounce, two-zone model of mutual illumination. We test our approach in both simulation and experiment. In the case of one surface patch in a scene, when the color constancy problem has no solution, we are able to obtain promising results. Introduction It has been shown that the human visual system makes use of mutual illumination information in color perception [1, 2]. However, there has been little work in the computer vision literature which exploits mutual illumination to improve upon color perception. In this work, we address the problem of spectral color constancy while making use of mutual illumination information. The problem of spectral color constancy lies in computing a surface reflectance spectrum that is independent of the spectrum of light incident on the surface. Most color constancy approaches with interreflections use spectral models for the surfaces and the illuminant, while employing finite-dimensional linear model representations for these spectra. In [3], Funt et al. assumed a one-bounce, two-zone model of interreflections. The one-bounce model takes into account the light reflected off one surface that bounces onto the other. The two zones comprise that in the mutual illumination region, also called the edge, and that far from this region. By taking mutual illumination into account, they effectively added a sensor class to Maloney and Wandell’s approach [4]. This is important because one more basis function can then be used to model the surface spectrum, given the restrictions on the number of basis functions in Maloney and Wandell’s approach. Drew and Funt [5] extended Funt et al.’s [3] approach to account for multiple zones, using the radiosity method from [6]. They also proposed the variational approach [7] that does not assume diffuse-illumination, a twopatches limit, or locations with negligible interreflection as in the previous approaches. It still assumes a one-bounce model of interreflection, however. Harder [8] investigated interreflections while assuming known illumination in addition to Lambertian surfaces. All the mentioned approaches use three basis functions for each of the surface and illuminant spectra. The only exception is in Harder’s work [8] where the illuminant spectrum is assumed to be known, which is not the case in this paper. Such a small number of basis functions may not be enough to provide an accurate representation of surface spectra. Moreover, all these approaches require the database of surface and illuminant spectra to be specified in advance in order to obtain the basis functions for the linear model representations. Such a requirement places these approaches at a major disadvantage as databases might not be available in advance. Even if they are available, they might not be consistent with the data in a certain application. We propose a novel approach that estimates surface and illuminant spectra in the presence of interreflections given only camera sensor responses. This approach is based on the color constancy technique introduced in [9] in which the surface and illuminant spectra are represented by maximum entropy models, and therefore do not require a set of basis functions to be specified in advance. Maximum entropy models were successfully used to estimate Munsell patch reflectance spectra given only photoreceptor responses in [10]. The use of maximum entropy models was inspired by Jaynes, who stated that a physical quantity frequently observed in practice will tend to a value that can be produced in the largest number of ways [11]. In the case of physical processes representing spectra, many surfaces observed in our everyday-life surroundings have spectra of high entropy, as opposed to monochromatic surfaces which have low entropy spectra [10]. The illuminant spectra are also represented by maximum entropy models as they are observed in our everyday-life surroundings and therefore can be produced in a large number of ways [9]. In this paper, the proposed approach is explained and derived. To this end, diffuse-illumination and a Mondrian scene composed of matte, Lambertian surfaces are assumed. The light illuminating a Mondrian scene is assumed to be locally constant. This means that the spectral characteristics of the light vary slowly. As for the interreflections, a one-bounce, two-zone model is assumed. Next, the performance of the approach is analyzed in simulation and then in experiment. In particular, in the case of one surface patch in the scene, it is shown how exploiting interreflection information improves upon spectral estimation. Moreover, while, to the knowledge of the authors, none of the previous approaches provided surface or illuminant spectral estimates for real images, such spectra are shown in this paper. Maximum Entropy Spectral Based Color Constancy with Interreflections The proposed color constancy approach aims at recovering surface and illuminant spectra in the presence of interreflections given only camera sensor responses. We assume a one-bounce,